MEMORANDUM January 11, 2019 
TO: Board Members 


FROM: Grenita Lathan, Ph.D. 
Interim Superintendent of Schools 


SUBJECT: TEACHER INCENTIVE FUND STEM GRANT: PROGRAM EVALUATIONS 
CONTACT: Carla Stevens, 713-556-6700 


The fourth cohort of the Teacher Incentive Fund federal grant competition (“TIF4”) included 
special consideration for projects that would identify, develop, and utilize master teachers as 
leaders of STEM education (science, technology, engineering, and mathematics). In September 
2012, HISD was awarded a TIF4 grant for $15.9 million over five years. The TIF4 project 
schools were among the HISD schools serving grades K-8 with the highest student economic 
disadvantage and the most risk factors for chronic absenteeism. 


Attached are the three program evaluation reports associated with the TIF4 grant. A human 
capital approach to strengthening STEM education addressed the TIF4 project schools’ need for 
high-quality supports for student learning, and the systemic challenges to teacher retention, 
development, and recruitment in hard-to-staff subjects. The first report in this series provided a 
descriptive overview of the grant-funded activities and interventions unique to the TIF4 project 
schools, setting the context for a meaningful discussion of programmatic impact. 


The second report in the series addressed student outcomes for State of Texas Assessments of 
Academic Readiness (STAAR) Mathematics (grades three through eight) and STAAR Science 
(grades five and eight), during the grant period of 2012—2013 to 2016-2017. The TIF4 
programming produced substantive, statistically significant results for science and for secondary 
mathematics. 


Key findings include: 

e STAAR Science, Grades 5 and 8. Over the grant period, the cumulative impact of the TIF4 
program on Grade 5 Science was an increase in student achievement of about a fifth of a 
standard deviation (0.20 SD). The impact on Grade 8 Science was about a quarter of a 
standard deviation (0.24 SD). Both estimates are statistically significant, although the 
evidence in eighth-grade science is less compelling. With a fifth of a standard deviation of 
improvement, a student initially at the 50th percentile would improve to the 58th percentile. 

e STAAR Math, Grade 6. The point estimates suggest a cumulative impact over the grant 
period of about a fifth of a standard deviation (0.21 SD). These estimates were not 
considered statistically significant at conventional levels. 

e STAAR Math, Grades 7 and 8. Over the grant period, the cumulative impact of the TIF4 
program on Grade 7 Math was about half of a standard deviation of student achievement 
(0.49 SD). The impact on Grade 8 was about four-tenths of a standard deviation (0.39 SD). 
Both estimates were statistically significant at conventional levels. A half-standard-deviation 
increase would improve the achievement of a student at the 25th percentile to the 43rd 
percentile, or a student at the 50th percentile would then grow to the 69th percentile. 


STAAR Math, Grades 3 to 5. In grades three through five, the TIF4 program did not appear 
to have a large effect on mathematics achievement in any year of the grant period. 


The third and final report overviews the performance-based compensation strategies 
implemented through the TIF4 grant, as well as situates that work in the context of HISD’s 
challenges for teacher retention and mobility. 


Key findings include: 


The TIF4 schools paid out about ten $5,000 retention bonuses for each $10,000 recruitment 
bonus (178 Retention vs. 18 Recruitment). This suggests that effective math and science 
teachers at hard-to-staff HISD schools find retention bonuses to be meaningfully more 
compelling than larger recruitment bonuses. 

In Years Three, Four, and Five, the TIF4 schools retained 75 percent of their Effective and 
Highly Effective math and science teachers. 

During the grant period, HISD directed $3,330,781 of federal, state, and local resources into 
the ASPIRE Award at the TIF4 project schools. Over a thousand (1,012) ASPIRE Awards 
were paid to educators at the TIF4 campuses during this time. Every TIF4 school had at 
least one educator who received an ASPIRE Award during the grant. 

By the start of the third year after their initial hire, 46 percent of new teachers had left the 
HISD school where they started. This attrition rate is higher for new math (60.8 percent) and 
new science (61.2 percent) teachers. 

During this period, the top ten percent of HISD schools (90th percentile and upward) 
annually retained over 80 percent of all their high TADS teachers, regardless of subject area 
or years of experience. 


Taken together, these findings strongly suggest that the high turnover among HISD’s math and 
science teachers can be mitigated through investment in retention bonuses for effective and 
highly effective teachers already working at specific campuses. 


Should you have any further questions, please contact Carla Stevens in Research and 
Accountability at 713-556-6700. 


Lora, AV ox 
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Teacher Incentive Fund STEM Grant in Houston ISD: 
A Human Capital Approach to Improving STEM Education 


Executive Summary 

Program Description 

The fourth cohort of the Teacher Incentive Fund federal grant competition (“TIF4”) included special 
consideration for projects that would identify, develop, and utilize master teachers as leaders of STEM 
education. In September 2012, HISD was awarded a TIF4 grant for $15.9 million to implement a human 
capital approach to improving STEM education. The TIF4 project schools were among the HISD schools 
serving grades K—8 with the highest student economic disadvantage and the most risk factors for chronic 
absenteeism. A human capital approach to strengthening STEM education addressed the TIF4 project 
schools’ need for high-quality supports for student learning, and the systemic challenges to teacher 
retention, development, and recruitment in hard-to-staff subjects. 


The first report in this series provided a descriptive overview of the grant-funded activities and interventions 
unique to the TIF4 project schools, setting the context for a meaningful discussion of programmatic impact. 
The second report in the series addressed student outcomes for State of Texas Assessments of Academic 
Readiness (STAAR) Mathematics (grades three through eight) and STAAR Science (grades five and eight), 
during the grant period of 2012-2013 to 2016-2017. This report overviews the performance-based 
compensation strategies implemented through the TIF4 grant, and situates the TIF4 schools in the context 
of HISD’s historic challenges for new teacher retention, effective teacher retention, and the retention of 
math and science teachers. 


Highlights 

Key findings in this third report include: 

e Overall, HISD paid about ten $5,000 teacher retention bonuses for each $10,000 teacher recruitment 
bonus (178 Retention vs. 18 Recruitment). In Years Three, Four, and Five, the TIF4 schools retained 
75% of their Effective and Highly Effective math and science teachers. 

e During the grant period, HISD directed $3,330,781 of federal, state, and local resources into the 
ASPIRE Award at the TIF4 project schools. Over a thousand (1,012) ASPIRE Awards were paid to 
educators at the TIF4 campuses during this time. Every TIF4 school had at least one educator who 
received an ASPIRE Award during the grant. 

e By the start of their third year, 46% of new teachers had left the HISD school where they were initially 
hired. This attrition rate is higher for new math (60.8%) and new science (61.2%) teachers. 

e During this period, the top ten percent of HISD schools (90" percentile and upward) annually retained 
over 80% of all their Effective and Highly Effective teachers, regardless of subject area or years of 
experience. 


This suggests that effective math and science teachers find retention bonuses to be meaningfully more 
compelling than recruitment bonuses that are twice as expensive and require a longer time commitment. 
Critically, these are teachers who have already shown success in meeting the needs of students at HISD’s 
hard-to-staff schools. Taken together, these findings strongly suggest that the high turnover among HISD’s 
math and science teachers can be mitigated through investment in retention bonuses for effective and 
highly effective teachers already working at specific campuses. 
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Introduction 


Since established by an Appropriations Act in 2006, the Teacher Incentive Fund (TIF) competitive grant 
program in the U.S. Department of Education (USDE) has supported human capital strategies “to ensure 
that students attending high-poverty schools have better access to effective teachers and principals, 
especially in hard-to-staff subject areas” such as science and math. Responding to the national agenda to 
improve STEM education, in 2012, the fourth cohort of the Teacher Incentive Fund federal grant competition 
(TIF4) included special consideration for projects designed to improve STEM education by identifying, 
developing, and utilizing master teachers as leaders of broader improvements (OESE, 2012a). 


In September 2012, Houston Independent School District (HISD) was awarded a TIF4 grant for $15.9 
million over five years (Award #S374B120011). The human capital strategies supported through TIF4 in 
Houston continue the successes and strategies of HISD’s previous TIF grants (Price & Stevens, 2017), and 
resemble the strategies undertaken by the other 35 TIF4 grant recipients nationwide (Oll, 2015). For more 
information about the Teacher Incentive Fund grant, see Appendix A (p. 25). 


HISD was one of six TIF4 grantees to support a “comprehensive approach to improving STEM instruction” 
as part of their overall human capital strategy (OESE, 2012b). These STEM-specific TIF4 grants were 
frequently described by USDE staff as TIF4-STEM grantees. A human capital approach to strengthening 
STEM education addressed the project schools’ need for high-quality supports for student learning, and the 
systemic challenges to teacher retention, development, and recruitment in hard-to-staff subjects. Through 
the TIF4 grant, HISD supported some activities that addressed teaching and learning across all content 
areas, and some activities that addressed teaching and learning only within the STEM content areas. 


This report is the third in a series, each assessing an aspect of the TIF4 programming at the project schools. 
The first report provided a descriptive overview of activities and interventions unique to the TIF4 project 
schools, setting the context for a meaningful discussion of programmatic impact (Price, Provencher, & 
Stevens, 2018). Under the assumptions guiding the TIF grant program, student outcomes are a function of 
human capital management inputs — educator recruitment, retention, selection, assessment, professional 
development and supports, and performance-based compensation (Miller et al., 2015) — as mediated by 
teaching and learning behaviors. Therefore, the second report addressed student outcomes for the State 
of Texas Assessments of Academic Readiness, or STAAR, Mathematics (grades three through eight) and 
STAAR Science (grades five and eight), during the grant period of 2012—2013 to 2016-2017 (Price, 
Christian, & Stevens, 2018). 


This report overviews the performance-based compensation strategies implemented through the TIF4 
grant, as well as situates that work in the context of HISD’s challenges for teacher retention and mobility. 
Several factors inform a teacher’s decision about where and what to teach — compensation, working 
conditions, and student characteristics such as race, prior achievement, and economic disadvantage status 
(Hanushek, Kain, & Rivkin, 2002; Hanushek & Rivkin, 2007). Among the major factors influencing teachers’ 
decisions in the labor market, compensation is one of the inputs that can be manipulated programmatically 
by school or district leadership. Performance-based compensation systems (abbreviated to PBCS in TIF 
grant documentation) are designed to recognize and financially reward teachers for student metrics 
associated with their instruction. Compensation strategies are an important component of achieving the 
district's goals around teacher retention and development. 


Within education, it is generally accepted that on average, a more experienced teacher should be 
considered a more effective educator than a teacher with less experience (Hanushek, Kain, & Rivkin, 2004). 
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In this context, the exit of an experienced teacher from a specific school will ripple outward to other schools: 
if experienced teachers do not stay in HISD, then they are often replaced by less-experienced or new 
teachers. These new teachers, in turn, may not remain in HISD long enough to become experienced 
teachers themselves, resulting in a cycle where even the highest-need students may frequently be taught 
by individuals who do not have sufficient expertise to meet their instructional needs. The final components 
of this report delve into these specific outcomes: new teacher retention, effective teacher retention, and the 
retention of math and science teachers. 


Methods 


In July 2012, HISD leadership identified specific schools to receive STEM programming through the TIF4 
grant (HISD, 2012). Located in almost every quadrant of Houston (See Appendix Figure 1, p. 26), each 
year, these schools served approximately 7,500 students from pre-kindergarten through eighth grade. Like 
most of the schools in HISD, the TIF4 project schools were considered “high-need” under the definitions in 
the U.S. Department of Education’s Request for Application (OESE, 2012a). Additionally, the TIF4 project 
schools each had a persistent track record of underperforming on the science STAAR exams required 
under the Elementary and Secondary Education Act (NCLB, 2002). Their inclusion in the TIF4 grant was 
intended to address student learning and achievement in both math and science. The TIF4 project schools 
were identified based on their need for supports, rather than randomly. Consequently, HISD project staff 
were precluded from conducting randomized controlled trials, which is considered the most rigorous 
research design for making causal inferences (Murnane & Willett, 2011). 


As a result, these analyses serve as descriptive models of historical behavior, and as explorations of 
existing trends. None of these analyses are appropriate for supporting causal inference statements about 
the impact of the TIF4 grant. The datasets, methodology, limitations, and findings for each of these 
components are described in each section below. 


Performance Based Compensation at TIF4 Schools 


As part of the grant, specific performance-based compensation strategies for recruitment and retention 
were implemented for STEM teachers at the TIF4 campuses. These represented a major investment of 
public resources, and they were an important aspect of the comprehensive TIF4A-STEM programming at 
the project schools that yielded such remarkable effects for student achievement in math and science as 
presented in the second report of this series. 


ASPIRE Award 

In January 2007, the Houston Independent School District (HISD) inaugurated the Teacher Performance- 
Pay Model 2005-2006, becoming the first school district in the nation to implement a performance-pay 
system of this magnitude based on individual teacher effectiveness. The experience gained in the first year 
and consultations with national experts and teachers provided the impetus for recommending the 
improvement and enhancement of the model, which became the ASPIRE Award under the “Recognize” 
component of the district's comprehensive education-improvement model, Accelerating Student Progress 
Increasing Results and Expectations (Hui, Mosier, & Bigner, 2018). The HISD Research and Accountability 
team published an annual program overview and evaluation of the ASPIRE Award during the TIF4 grant 
period (Zimmerman, Hui, & Mosier, 2017a, 2017b; Zimmerman, Hui, Mosier, & Chang, 2015; Zimmerman, 
Mosier, & O’Brien, 2014). From 2007 through 2016, the ASPIRE Award program was available to 
educators, school leaders, and support staff at all HISD schools. The model underwent updates over the 
years, staying relevant to changes in state assessments, state accountability measures, and available 
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metrics of teacher performance. For the 2016-2017 school year, this Award was available only to educators 
and school leaders at HISD schools participating in the TIF4 grant. 


The outcomes of the ASPIRE Award program at the TIF4 schools can be measured three ways: in 
cumulative number of awards, in the consistency with which a school had an award paid, and in the sum 
of performance-based compensation received by educators at TIF4 project schools. 


While the schools represent a wide range of performance along this metric, every TIF4 school had at least 
one educator who received an ASPIRE Award during the grant period. Over a thousand (1,012) ASPIRE 
Awards were paid to educators at the TIF4 campuses during this time. This represents an average of 44 
Awards per school. As shown in Figure 1 (p. 7), nine schools saw a more-than-average number of Awards 
during this period: Garden Oaks Montessori (105), Braeburn ES (102), Southmayd ES (100), Fondren MS 
(95), Eliot ES (83), Herrera ES (78), Wilson Montessori (67), Burrus ES (50), and Fleming MS (50). The 
other schools had fewer Awards than the group average of 44. (See Appendix Table 1, p. 27, for details.) 


The TIF4 grant period covered five ASPIRE Awards. On average, a specific school saw an ASPIRE Award 

in 3.7 years, or 75% of the time. In none of these five years did an educator at every TIF4 school earn an 

Award; the number of schools with an Award ranged from 14 schools for 2016-2017 to 20 schools for 

2014-2015. A measure of a school’s consistency in earning ASPIRE Award-level-metrics, then, is the 

fraction of these five years in which at least one their educators earned an ASPIRE Award. Thirteen schools 

were above this group average. 

e At 100%, 11 Schools with Awards in five years: Braeburn ES, Burrus ES, Eliot ES, Fleming MS, 
Fondren MS, Foster ES, Garden Oaks Montessori, Herrera ES, Southmayd ES, Sugar Grove 
Academy, Wilson Montessori 

e At 80%, two schools with Awards in four years: Durkee ES and Law ES 

The remaining ten project schools were below the 75% group average in terms of consistency. Not 

surprisingly, many of the schools whose educators consistently earned an ASPIRE Award are also above 

the group’s per-school average of 44 Awards during the project period. The notable exceptions with below- 
average performance and above-average consistency are Foster ES (32 Awards in 5 years), Looscan ES 

(24 Awards in 5 years) and Sugar Grove Academy (14 Awards in 5 years). 


As illustrated in Figure 2 (p. 7), cumulative ASPIRE Award payouts at each school over the five-year grant 
period range from $403,350 and $330,854 (Fondren MS and Southmayd ES, respectively) to $16,500 and 
$3,750 (Codwell ES and Mading ES, respectively). Details on each school’s payout for each year can be 
found in Appendix Table 2 (p. 28). The HISD Research and Accountability team published extensive 
details on the Awards payout for each year of the TIF4 grant (Hui & Carney, 2016; Hui & Mosier, 2015; Hui, 
Mosier, & Bigner, 2017, 2018; Mosier & LaSage, 2014). The 2018 analysis covered exclusively the TIF4 
project schools. 


During the grant period, HISD directed $3,330,781 of federal, state, and local resources into the ASPIRE 
Award at the TIF4 project schools. As shown in Figure 3 (p. 8), these resources varied by Award year — 
from $265,625 in Year Four to $933,508 in Year Two. Aggregated across the five ASPIRE Award payouts, 
three percent came from state funds ($100,517 from the District Awards for Teacher Excellence Program, 
or DATE, for Award 2012-2013), 31.2% from federal funds ($1,038,467 from TIF4), and 65.8% from local 
district funds ($2,191,797 from HISD’s general fund). This breakdown is cumulative across all five years, 
and it does not reflect the sum of HISD’s locally funded investment in the TIF4-STEM project. 
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Figure 1. Cumulative Count of ASPIRE Awards at TIF4 Project Schools, 2012-2013 to 2016-2017 
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Figure 2. Cumulative ASPIRE Award Payout at TIF4 Project Schools, Awards 2013-2017 
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Figure 3. Cost Division of ASPIRE Awards at TIF4 Project Schools, Awards 2012-2017 
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Note: These totals include neither the fringe benefits on this compensation, nor the payout of ASPIRE Awards to campus- 
based support staff who were not eligible for TIF4 funding. See Appendix B (p.27). 


Recruitment and Retention Bonuses for STEM Teachers at TIF4 Project Schools 

As noted above, HISD was one of just six TIF4 grantees to support a “comprehensive approach to improving 
STEM instruction” as part of their overall human capital strategy (OESE, 2012b) — including recruitment 
and retention bonuses paid to qualifying math and science teachers. 


Teachers already at a specific TIF4 project school were eligible for a retention bonus of up to $5,000 if the 

teacher returned to the same TIF4 campus by the first duty day of the following fall semester; and, 

e The teacher was scheduled to teach core foundation courses“) for either math or science or both in a 
STAAR or End of Course (EOC) tested grade and subject for the following academic year; and, 

e The teacher was considered Effective or Highly Effective according to the HISD Teacher Appraisal and 
Development System (TADS) for the most recently available data; this summative rating must include 
measures of Student Performance; and, 

e The teacher had a Student Growth (Education Value-Added Assessment System, or EVAAS) 
Cumulative Gain Index score for either math or science at or above 1.0 for the most recently available 
academic year in a STAAR/EOC tested grade and subject; or, the teacher has at least one teacher- 
level Comparative Growth metric in the top two quintiles for math or science for the Academic Year 
2015-2016 in a STAAR/EOC tested grade and subject (Year Five only). 

Teachers working in HISD at a school other than a TIF4 campus were eligible for a Recruitment bonus of 

up to $10,000 by (a) moving to a TIF4 campus as of the first duty day of the following academic year, (b) 

meeting the three criteria outlined above, and (c) making a verbal commitment to work at the campus for a 

five-year period. 


Due to the timing of metrics availability vis-a-vis the hiring cycle, new science and math teachers would not 
have the student-level data required for bonus eligibility until after the conclusion of their second full year 


in the classroom. To illustrate: in March 2016, HISD administrators identified and notified those HISD 
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teachers whose documented effectiveness and evaluation criteria indicated they would be eligible for a 
bonus should they return to their TIF4 campus to teach for the 2016-2017 school year. These teachers 
were identified before TADS or EVAAS metrics for the current school year were completed, but in time to 
be relevant to their decisions about where to teach in the following school year. Consequently, the bonuses 
paid in September 2016 were based on the most recent data available at the time of the identification in 
March 2016: data from 2014—2015 (finalized in December 2015 for all HISD teachers). Performance metrics 
from the 2015-2016 school year drove the bonuses paid to teachers identified in March 2017, for bonuses 
to be paid in September 2017 in support of staffing for the 2017-2018 school year. 


In the first two annual cycles of the bonus program, a total of five recruitment and 46 retention bonuses 

were paid through the TIF4 grant (Figure 4). These numbers were meaningfully higher in Years Three, 

Four, and Five: 

e In September 2015, HISD paid six Recruitment and 41 Retention bonuses to eligible TIF4 teachers, or 
retention of 74.5% of the 55 eligible teachers. 

e In September 2016, HISD paid six Recruitment and 47 Retention bonuses to eligible TIF4 teachers, or 
retention of 85.5% of the 55 eligible teachers. 

e In September 2017, HISD paid one Recruitment and 44 Retention bonuses to eligible TIF4 teachers, 
or retention of 74.6% of the 59 eligible teachers. 


Overall, HISD paid about ten retention bonuses for each recruitment bonus (178 vs. 18). Project staff have 
attributed the relative later success to the realignment of the cycle calendar that took place early in Year 
Three. Principals at project schools received the names of teachers who would be eligible for a retention 
bonus if they returned for the following year, as well as the names of all HISD teachers who would be 
eligible for a recruitment bonus if they were successfully recruited for the following year. Additionally, the 
teachers currently at TIF4 schools received a communication directly notifying them of their bonus eligibility, 
should they return for the following year. (See Appendix C, p. 29, for a sample.) In Year Three, this direct 
notification went out in May, in Year Four in April, and in Year Five in March. 


Figure 4. TIF4 Recruitment (18) and Retention (78) Bonuses, September 2013 to September 2018 
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Teacher Retention and Mobility During the TIF4 Grant Period 


How long do new teachers stay in HISD? 

One of the goals of the TIF4 grant was to improve district-level human capital management systems — 

consequently, an assessment of district-wide measures is necessary to contextualize the human capital 

outcomes of the specific TIF4 schools. Project staff conducted a descriptive analysis that followed three 

cohorts of first-year") teachers in HISD, from their entry into HISD during the 2012-2013, 2013-2014, and 

2014-2015 school years. These cohorts of first-year teachers were followed from their first fall semester 

until the beginning of the 2018-2019 school year. Cohort 1 (originally n=1,581) first entered HISD in 2012-— 

2013, Cohort 2 (originally n=1,666) first entered HISD in 2013-2014, and Cohort 3 (n=1,757) first entered 

HISD in 2014—2015. This gives a district-level perspective on the three types of movement available to 

newly hired teachers: 

e Returned to Same School Next Year: These individuals were listed as teachers on the same HISD 
school’s employee roster in the fall semester of the following academic year. 

e Moved Schools for Next Year: These “mover” individuals were listed as teachers on a different HISD 
school’s employee roster in the fall semester of the following academic year. 

e Left District for Next Year: These “leaver” individuals were not listed as teachers on any HISD school’s 
employee roster in the fall semester of the following academic year. 


This methodology does obscure those movements that are the result of career development — for example, 
a teacher who takes on a new role at the same campus as an Assistant Principal is treated as a “leaver”. 
This analytical choice was deliberate because from the district-wide perspective, the promotion creates a 
vacancy in an instructional position that would otherwise have been filled; even though the promotion may 
be a positive development for the teacher and for the campus, it is a negative development for the district- 
wide measures. This is one of the ways in which district-level goals for teacher retention and career 
development can be in tension with campus-level goals. 


Table 1 (p. 11) shows how many cohort-member teachers were employed at the start of each year, and 
where they appeared in the following year’s staffing data. This table can be interpreted as follows: for Cohort 
1 in 2012-2013, there were 1,581 first year teachers in HISD. For example, for the year following their first 
full year of teaching, 1,209 teachers from Cohort 1 returned to teach in the same HISD school, 105 moved 
to a different school in HISD, and 267 were not employed as teachers on the roster of any HISD school. 


These data can also be used to calculate the percentage of teachers in each cohort who returned in the 

following year to the school where they were initially hired. Table 2 (p. 11) shows the cohorts’ same-school 

retention for each additional year of their career. For example, 76.47% of teachers hired in the 2012-2013 

cohort returned to teach in the same school in 2013-2014. This analysis presents HISD with several trends: 

e One year after hiring, an average of 72% of teachers returned to the school where they were initially 
hired. Inversely, by the start of their second year, nearly 30% of teachers had left the school where they 
were initially hired. 

e Two years after hiring, an average of 54% of teachers returned to the school where they were initially 
hired. Inversely, by the start of their third year, 46% or nearly half of teachers had left the school where 
they were initially hired. 

e On average, a school that hires a teacher with no years of teaching experience who is also new to 
HISD has just slightly better than a 50/50 chance of benefiting from that teacher's third year of teaching. 

e Across all three cohorts, teacher mobility retention leveled out four years after the initial hire. 

These results are presented graphically below in Figure 5 (p. 12), outlining the magnitude of the challenge 

at hand. 
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Table 2 (below) and Figure 5 (p. 12) do not make the distinction between a teacher's promotion, a teacher’s 
leave of HISD, and a teacher's movement to another school within HISD. While these are relevant 
distinctions for district-level objectives, from the school’s perspective, they are all equally problematic since 
they each create a vacancy in an instructional position that would otherwise have been filled. This is another 
example of how campus-level and district-level goals for teacher retention and career development are not 
always in tight alignment. 


Table 1. Retention and Movement in HISD for Three Cohorts of New Teachers 


bTeq aloze) | Conant Count at Returned to Same Moved Schools Left Tchr. Roster 
Year Beginning of Year School Next Year for Next Year for Next Year 
2012-2013 1 iste 1,209 105 267 
2013-2014 4 1,274 874 111 289 
2014-2015 1 930 657 94 179 
2015-2016 1 753 594 59 100 
2016-2017 4 672 522 54 96 
2017-2018 1 572 468 34 70 
2013-2014 2 1,666 1,158 120 388 
2014-2015 2 1,205 859 116 230 
2015-2016 2 973 aly 70 186 
2016-2017 2 785 596 70 119 
2017-2018 2 662 505 63 94 
2014-2015 3 1,757 1,224 162 371 
2015-2016 3 1,296 968 79 249 
2016-2017 3 1,045 776 73 196 
2017-2018 3 834 645 68 2 


Table 2. Same-School Retention of New Teachers in Three Cohorts, By Year 


Years After Hire Average Cohort Cohort 1 Cohort 2 Cohort 3 
SS Retention (2012-2013) (2013-2014) (2014-2015) 
1 Year After Hire 72.0% 76.47% 69.51% 69.66% 
(a.k.a., start of 2™ year) j (1,209/1,581) (1,158/1,666) (1,224/1,757) 
2 Years After Hire 54.0% 55.28% 51.56% 55.09% 
(a.k.a., start of 3 year) ‘ (874/1,581) (859/1,666) (968/1,757) 
3 Years After Hire 42.9% 41.56% 43.04% 44.17% 
(a.k.a., start of 4" year) ; (657/1,581) (717/1,666) (776/1,757) 
4 Years After Hire 26.7% 37.57% 35.77% 36.71% 
(a.k.a., start of 5" year) ; (594/1,581) (596/1,666) (645/1,757) 
5 Years After Hire 31.7% 33.02% 30.31% -- 
(a.k.a., start of 6" year) ; (622/1,581) (505/1,666) 
6 Years After Hire 29.60% 


(a.k.a., start of 7" year) 
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Figure 5. Within-School Retention of New Teachers in HISD, for Three Cohorts 
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How long do new math and science teachers stay in HISD? 

Research on the STEM teaching workforce has suggested that math and science teaching positions are 
more difficult to staff than other subject-specific roles. Each year of the TIF4 grant period, the Texas 
Education Agency (TEA) supported this conclusion by formally naming mathematics and science as 
“teacher shortage” areas (TEA, 2012, 2013, 2014, 2015, 2016). The next step of analysis repeated the 
analysis described above, focusing on new math and science teachers, districtwide. 


Initial screening for math and science teachers relied solely on job coding in HISD’s human resources 
information system. However, this excluded multi-subject teachers from elementary grades, and over- 
reported the outcomes of secondary math and science teachers. Ultimately, these teachers were identified 
by the content area of their courses, as listed in the district's student information system (Chancery). This 
approach did not exclude elementary teachers, and therefore provided a more comprehensive view of 
STEM instruction across the district than the initial strategy. There were two tradeoffs: (1) teachers assigned 
math and science courses (e.g., self-contained) were counted in both analyses, even though they only 
occupy one position, and; (2) the final course assignments for spring semester of 2018-2019 were not yet 
available, and so the overall analysis period excludes 2018—2019. 


Overall, the retention for new math and science teachers is meaningfully less successful than the district- 
wide average for all subjects (outlined in previous section). The numbers in Table 3 (p. 13) and Table 4 (p. 
13) show the percentage of new math and science teachers who returned to the same school and subject. 
Each year, the average cohort retention rate for math and for science teachers is nearly 10 percentage 
points lower than the cohort’s overall same-school retention rate. Three years after hiring, a little over a 
quarter of the cohort’s initial math teachers are still teaching math at their initial school. The rates are similar 
for new science teachers: less than a third are still teaching science at their original school. 


However, even if they stay in the same school, it is possible that a person who is teaching math in their first 


year is teaching other subject areas in their later years. The school benefits from retaining the teacher in a 
different role, even if this mobility is considered a loss for subject-specific retention. Consequently, project 
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Table 3. Same-School and Same-Role Retention of New Math Teachers, by Cohort/Year 


Years After Hire Average Cohort fo) Te) amet Cohort 2 Cohort 3 
Retention (2012-2013) (2013-2014) (2014-2015) 
1 Year After Hire 57.8% 62.91% 54.59% 56.00% 
(a.k.a., Start of 2" year) j (380/604) (357/654) (392/700) 
2 Years After Hire 39.2% 42.22% 35.47% 39.86% 
(a.k.a., Start of 3 year) (255/604) (232/654) (279/700) 
3 Years After Hire 28.8% 26.99% 28.13% 31.14% 
(a.k.a., start of 4'* year) ‘ (163/604) (184/654) (218/700) 
4 Years After Hire 23.3% 23.68% 22.94% _ 
(a.k.a., start of 5" year) ‘ (143/604) (150/654) 
5 Years After Hire 22.85% 
(a.k.a., start of 6" year) (138/604) _ ~ 


Table 4. Same-School and Same-Role Retention of New Science Teachers, by Cohort/Year 


Years After Hire Average Cohort Cohort 1 of) Te) a ar Cohort 3 
Retention (2012-2013) (2013-2014) (2014-2015) 
1 Year After Hire 57.7% 61.43% 55.27% 56.37% 
(a.k.a., start of 2™ year) (336/547) (341/617) (376/667) 
2 Years After Hire 38.8% 40.77% 36.14% 39.58% 
(a.k.a., start of 3 year) (223/547) (223/617) (264/667) 
3 Years After Hire 29.0% 27.79% 29.01% 30.28% 
(a.k.a., start of 4" year) (152/547) (179/617) (202/667) 
4 Years After Hire 29.0% 23.95% 21.07% 7 
(a.k.a., start of 5" year) (131/547) (130/617) 
5 Years After Hire 19.56% 
(a.k.a., start of 6" year) - (107/547) - ia 


staff looked at both kinds of retention for math and science teachers: same-school and same-role, as well 
as same-school and different role. Figure 6 (p. 14) shows the same-school retention of those individuals 
who taught math in their first year who taught math in the following years, while Figure 7 (p. 14) shows the 
same-school same-role retention of new science teachers. In both, the rate shown is for Cohort 1. The 
rates of same-school retention regardless of content area are described in Appendix Tables 3-4 (p. 30). 


Same-school retention for new teachers is relatively low, but this is also the case for more experienced 
teachers. We also calculated the average number of years any HISD teacher who appeared in the data 
window stayed at any one school during that time. As shown in Table 5 (p. 14), the average duration of a 
teacher's stay at one school over the five-year period was less than three years (2.51), and lower for math 
and science teachers. Note that this does not mean the average teacher in this window left in the middle of 
their second year. Rather, it means that the average teacher left either two or three years after initial hiring 
(for both math and science, more often two years rather than three). 


Since project staff did not incorporate information on where the experienced teachers were employed 
before the beginning of the observation period, these figures underestimate the true total duration of the 
average HISD teachers’ time in one school. However, this does suggest that the average HISD teacher 
moves relatively frequently, within a given five-year period, and therefore suggests that the stability of the 
faculty at the typical HISD campus is relatively low. This may hinder a school’s ability to build a cohesive 
faculty that functions as a professional learning community. 
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Figure 6. Same-School Same-Role Retention for Three Cohorts of New Math Teachers 


100.0% 
90.0% 
80.0% 
70.0% 
60.0% 
50.0% 
40.0% 
30.0% 
20.0% 
10.0% 

0.0% 


100.0% 


27.0% 
23.7% 22.8% 


Baseline Year 1 Year 2 Year 3 Year 4 Year 5 


=e—Cohort1 —™Cohort2 —t Cohort 3 


Figure 7. Same-School Same-Role Retention for Three Cohorts of New Science Teachers 
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Table 5. Average Number of Years at Same School for HISD Teachers, 2012-2013 to 2017-2018 
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Which HISD schools are especially good at retaining their top teachers? 

Teacher turnover has been shown to be detrimental to the quality of instruction, especially in low performing 
schools (Hanushek, Rivkin, & Schiman, 2016; Ronfeldt et al, 2013). While turnover is likely to be somewhat 
of an issue in most schools, there may be some schools in HISD that are substantially better at retaining 
the most effective teachers, even after controlling for those factors outside schools’ control that influence 
teacher turnover. The former can provide HISD leadership with possible models for emulation, while the 
latter could be targeted for additional support. This is important because not all turnover has the same 
impact on instruction — because “if effective teachers are less likely to leave than less effective teachers, 
however, then high levels of teacher attrition may improve rather than decrease the overall quality of the 
teaching workforce” (Guarino, Santibanez, & Daley, 2006). 


To identify which schools are better than expected at the task of retaining their effective and highly effective 
teachers, project staff first used five years of employee rosters, school demographics, and teacher appraisal 
(TADS) data to develop a quantitative model estimating the likelihood that a teacher is retained in that 
school the following year, given school (percent Hispanic, percent African American, percent economic 
disadvantage) and teacher (TADS rating, experience level) characteristics that are known from other 
research to influence teacher turnover (e.g., Borman & Dowling, 2008). This model estimated the unique 
contribution of the school to teacher retention after removing the effects of year to year variation“), and 
after controlling for the school’s student characteristics known from other research to influence teacher 
turnover — the school’s percentage of economically disadvantaged students, Black, Hispanic, Asian, and 
White students. 


The school’s random effect, then, is the school’s distance from the expected level of retention for schools 

with similar student populations. Model results were used to derive an estimate of how schools compared 

to each other in terms of retaining each of the four categories of teachers: New and High TADS, New and 

Low TADS, Experienced and HIGH TADS, as well as Experienced and Low TADS. (For definitions of each 

term, see Appendix E, p. 31.) This modeling showed that HISD schools do differ substantially in their 

retention of teachers within each category. Figure 8 (p. 16) shows the individual schools’ random effect on 

retention, for each of the teacher types: 

e Each school is represented on each panel by one dot. These dots are ordered by the size of the effect 
(vertical axis) and arranged visually from left to right according to their percentile rank (horizontal axis). 

e Schools to the right of the 50" percentile on the horizontal axis had a positive effect on teacher retention 
— meaning that they retained that category of teachers at a higher rate than what was expected. 

e Schools to the left of the 50" percentile had a negative effect on teacher retention — meaning that they 
retained that category of teachers at a lower rate than what was estimated for them. 

e At the 50‘ percentile, the school effect is equal to zero — meaning that those schools retained that 
category of teacher at the same rate that was estimated for a school with their student characteristics. 


Note that there is a greater difference between schools at or above the 75" percentile and those at or below 
the 25'" percentile in retention of low TADS teachers compared to high TADS teachers — in both rows, the 
dots in the right-hand graphic (High TADS) are more tightly clustered than the dots in the left-hand (Low 
TADS) graphics. 
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Figure 8. Same-School Retention for Four Teacher Categories, School Effect and Percentile Rank 
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Additional analysis showed substantial variation of teacher retention across schools at most levels of these 
school characteristics, especially at high percentages of economic disadvantage, all percentages of 
Hispanic students, and low percentages of African American students. These findings are illustrated in 
Appendix Figures 2, 3, and 4 (Appendix E, pp. 31-32). Schools whose dots are above the district-wide 
trend line are retaining teachers at a higher rate than expected for that demographic factor; schools below 
the trend line are retaining teachers at a lower rate than expected. 


The district-wide trend line in Appendix Figure 2 (p. 31) shows regardless of whether they are low or high 
TADS, new teachers are increasingly unlikely to be retained as economic disadvantage rate increases 
(negative slope), whereas this factor appears to have almost no relation to the retention of experienced 
high TADS teachers (minimal or totally flat slope). The district-wide trend line in Appendix Figure 3 (p. 32) 
shows that the retention of teachers becomes less likely with the increase of each school’s percentage of 
Hispanic students. This is true for all four categories of teachers (four negative slopes). 


The district-wide trend lines in Appendix Figure 4 (p. 32) show a different pattern: the school’s percentage 
of African-American students has a weak relationship to the retention of new and low TADS teachers 
(minimal or totally flat slope), and a positive relationship to the retention of the three other categories of 
teacher (three positive slopes). 


The left-hand columns of Table 6 (p. 17) show the eight TIF4 project schools that are above the district 
average (50' percentile) for their retention of experienced high TADS teachers, as ranked by their 
estimated school effect on retention. The right-hand columns of the table show the six TIF4 project schools 
that are above the district average (50" percentile) for their retention of new high TADS teachers, when 
ranked by their estimated school effect on retention. These schools are represented among the dots above 
the 50" percentile line in the two High TADS graphs in Figure 8 (above). 
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Table 6. Retention of High TADS Teachers: TIF4 Schools Above HISD Average (50 Percentile 


Experienced High TADS Teachers New High TADS Teachers 

TIF4 School Effect Percentile TIF4 School Effect Percentile 
Burrus ES 0.043 80.7% Ross ES 0.068 92.7% 
Foster ES 0.035 76.0% Fleming MS 0.047 84.9% 
Ross ES 0.029 72.4% Foster ES 0.045 84.4% 
Law ES 0.029 71.9% Wilson ES 0.025 70.3% 
Braeburn ES 0.028 71.4% Burrus ES 0.011 57.8% 
Codwell ES 0.027 70.3% Codwell ES 0.008 54.7% 
Wilson ES 0.013 59.9% 

Blackshear ES 0.006 54.7% 

Table 7. HISD’s Average Estimated Retention Rate by Year, TADS Rating, and Teacher Experience 
Year Experienced Teachers New Teachers | Experienced Teachers New Teachers 
2012-2013 0.68 0.69 0.82 0.75 
2013-2014 0.64 0.53 0.80 0.72 
2014-2015 0.57 0.55 0.79 0.73 
2015-2016 0.60 0.56 0.82 0.76 
2016-2017 0.63 0.59 0.82 0.75 


Table 7 (above) shows the average estimated retention rate by low and high TADS and experienced and 
new teachers for each year. Overall, the average retention rate is higher for high TADS teachers in all years 
for both experienced and new teachers. Between experienced and new teachers, there are no measurable 
differences in the estimated retention rate between low and high TADS teachers. Additionally, the estimated 
retention rates do not vary significantly by year. 


Which HISD schools are especially good at retaining their math and science teachers? 

Earlier analysis of teacher mobility (reported above) required the identification of each school’s average 

annual retention of effective and highly effective teachers across all subject areas during the five-year 

observation window. (See Appendix E for definitions.) This also yielded numbers for each school’s average 

annual retention rates for effective and highly effective math and science teachers. In Figure 9 (p. 18), 

these rates have been sorted from low to high (vertical axis), ranked as percentiles (horizontal axis), and 

plotted close together. The resulting dots illustrate the distribution of retention rates across the district: 

e During this period, the top ten percent of HISD schools (90" percentile and upward) annually retained 
over 75% of all their high TADS teachers, regardless of subject area or years of experience. 

e Nearly two-thirds (64%) of HISD’s schools demonstrated an annual average retention over 50% for all 
six groups of high TADS teachers. 


In most of HISD’s schools, new and high TADS teachers were retained at a lower rate for each of the 
subject areas than their more experienced counterparts (the light lines are lower than the dark lines for all 
three subjects until near the 70% mark). This lends further support to the conclusion suggested by Table 
5: even within the overall challenge of turnover, a school is more likely to experience turnover of their High 
TADS teachers for math and science than for a general subject. 
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Figure 9. Historical Retention of New and Experienced Teachers, Comparing Across Subjects 
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As with the retention of all teachers, the retention of both math (Appendix Figure 5, p. 32) and science 
(Appendix Figure 6, p. 33) teachers showed significant variation in the lowest and highest school effects. 
Again, while student demographics influenced retention, as shown in Appendix Figures 7-12 (pp. 34—36), 
there was considerable variation in the retention estimates especially at high levels of school percentage 
of free/reduced price lunch, low levels of the percentage of African American students, and variation at all 
levels of Hispanic students. This again suggests that some schools are better at retaining High TADS math 
and science teachers even when school demographics are similar. 


Table 8 (below) shows the nine TIF4 schools above the district average for their retention of experienced, 
high TADS mathematics teachers, as ranked by their estimated school effect on retention. The right-hand 
columns of the table show the five TIF4 project schools above the district average for their retention of new 
high TADS mathematics teachers, also ranked by school effect. Table 9 (p. 19) shows the same for science. 


Table 8. Retention of High TADS Mathematics Teachers: TIF4 Schools Above 50‘ Percentile 


Experienced High TADS Math Teachers New High TADS Math Teachers 
TIF4 School Effect Percentile TIE4 School Effect Percentile 
Eliot ES 0.106 85.0% Eliot ES 0.140 94.7% 
Codwell ES 0.093 82.9% Southmayd ES OMA 89.8% 
Garden Oaks 0.067 74.9% Ross ES 0.081 80.2% 
Fondren MS 0.053 67.9% Garden Oaks 0.069 76.5% 
Sugar Grove Aca. 0.045 64.7% Fondren MS 0.063 74.3% 
Ross ES 0.038 61.5% 

Blackshear ES 0.036 59.4% 
Southmayd ES 0.009 55.1% 
Braeburn ES -0.003 50.3% 
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Table 9. Retention of High TADS Science Teachers: TIF4 Schools Above 50" Percentile 


Experienced High TADS Science Teachers New High TADS Science Teachers 
TIF4 School Effect Percentile TIF4 School Effect Percentile 
Fleming MS 0.124 87.0% Eliot ES 0.125 95.1% 
Sugar Grove Aca. 0.109 83.7% Sugar Grove Aca. 0.107 88.6% 
Codwell ES 0.107 81.5% Fleming MS 0.107 88.0% 
Ross ES 0.077 72.8% Garden Oaks 0.056 72.8% 
Garden Oaks 0.044 66.3% Ross ES 0.056 72.3% 
Burrus ES 0.013 Sele Burrus ES 0.048 67.9% 
Eliot ES 0.002 51.6% Herrera ES 0.047 67.4% 
Codwell ES 0.017 58.2% 
Southmayd ES 0.006 53.8% 


Fifteen of the 23 TIF4 project schools appear on at least one of these six “above average” lists. As a point 
of interest, participation in the TIF4 grant might be expected to cause retention of math and science teachers 
to vary even among schools with similar student demographics. Since the TIF4 project focused on STEM 
teaching, providing performance-based compensation and retention bonuses to math and science teachers 
in tested grades and subjects in the grants schools, we might expect that if these interventions influenced 
math and science teachers’ retention decisions, the TIF4 schools would have, on average, larger, more 
positive school effects. 


We found that the differences between the average school random effects on the retention rate for high 
TADS of TIF4 and non-TIF4 schools were small, and in favor of the TIF4 schools only for math and science 
teachers with three or fewer years of experience. Table 10 (below) shows the average school effects in 
percentage points for high TADS new and experienced teachers. A positive difference shows where TIF4 
schools on average had higher retention, a negative difference shows where TIF4 schools had lower 
retention than non-TIF4 schools. 


Note that this analysis is not a specific test of the effect of the TIF intervention on retention. An analysis at 
the teacher level (rather than the school level) might show different results for several reasons. First, 
schools are weighted the same regardless of size. Second, the retention rate includes both teachers that 
have received retention bonus or performance-based pay and those that did not. Third, this analysis does 
not control for other teacher-level characteristics that might affect retention, such as age or gender. While 
the present analysis does suggest that, over the years studied, TIF schools did not on average have very 
different retention rates than non-TIF schools, more analysis would be needed to estimate specific TIF4 
effects on individual teacher retention. 


Table 10. TIF4 and Non-TIF4 Average Effects on Retention: High TADS Math/Science Teachers 


TIF4 Schools Non-TIF4 Schools Difference 
New Math Teachers 0.1% -0.2% 0.3% 
Experienced Math Teachers -1.2% 0.2% -1.0% 
New Science Teachers 0.5% -0.2% 0.7% 
Experienced Science Teachers -2.2% 0.1% -2.3% 


Another point of interest is whether schools’ overall tendency to retain teachers varies by TADS ratings or 
teacher experience. First, we examined whether schools that tend to retain new teachers also retain 
experienced teachers. We found that there was a substantial correlation between schools’ relative effects 
on retention for new and experienced teachers, both high TADS and low. Table 11 (p. 20) shows the 
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correlations of school effects for new and experienced teachers, for all teachers, math teachers, and 
science teachers at both performance levels. The positive correlations show that on average schools that 
are good at retaining inexperienced high TADS teachers are also good at retaining experienced high TADS 
teachers. It also seems that schools that retain inexperienced low TADS teachers also retain experienced 
low TADS teachers. 


Table 11. Correlation of School Effects on the Retention of New and Experienced Teachers 


Correlation of School Retention Effects for New and Experienced Teachers 


High TADS Low TADS 
All Teachers 0.54 0.85 
Math Teachers ON2 0.60 
Science Teachers 0.73 0.53 


Another point of interest is whether schools’ overall tendency to retain teachers varies by TADS ratings. 
Ideally, schools that are good at retaining high TADS teachers would also be good at exiting low TADS 
teachers, so that in such schools, retention rates for low TADS teachers would be lower. Correlations 
between retaining high and low TADS teachers would thus be negative. We found that, overall, correlations 
were substantial and positive, as shown in Table 12 (below). 


These correlations suggest that schools that are better at retaining high TADS teachers are likely also to 
be better at retaining low TADS teachers. This suggests that some schools are generally more likely to exit 
less effective teachers, regardless of the experience. This seems especially true of new math teachers. It 
is possible that if schools have a difficult time finding qualified math teachers, they may find it more 
expedient to retain less effective teachers and try to develop them rather than to try to hire new ones that 
are more effective. 


Taken together, these correlations show that many schools are generally better or worse at retention, 
notwithstanding teacher effectiveness or experience. This seems especially true of retention of effective 
(high TADS) and less than effective (low TADS) math and science teachers. 


Table 12. Correlation of School Effects on the Retention of Low TADS and High TADS Teachers 


Correlation of School Retention Effects for High and Low TADS Teachers 


New Teachers Experienced Teachers 
All Teachers 0.62 0.64 
Math Teachers 0.88 0.80 
Science Teachers 0.70 0.80 
Conclusion 


HISD staff and school leadership are grateful for the investment of federal TIF4 resources that enabled 
teachers and students to experience a comprehensive approach to improving STEM education. While we 
recognize that there are areas for continued improvement when it comes to the retention of math and 
science teachers across all experience levels, the lessons learned through the implementation of the TIF4 
grant will continue to shape HISD’s strategies moving forward. In this context, the district can point to three 
specific takeaways from the human capital approach to improving STEM education. 


First, every TIF4 school had at least one educator who received an ASPIRE Award during the grant period. 
During the grant period, over a thousand ASPIRE Awards (1,012) were paid to educators at the TIF4 
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campuses. This illustrates HISD’s commitment to identify and recognize teachers whose students are 
exceeding expectations; it reflects the reality that high-quality teaching happens in every school across 
HISD. 


Second, for three consecutive years, the TIF4 schools retained 75% of their Effective and Highly Effective 
math and science teachers. Project staff attribute this in part to the realignment of the bonus calendar — 
by providing bonus-eligible teachers with relevant communications about their eligibility before most teacher 
choices were finalized. Additionally, anecdotal evidence collected by project staff suggests that teachers 
were also incentivized to stay in the TIF4 schools because of the meaningful STEM instructional resources 
and job-embedded professional supports provided through the grant. 


Third, across the grant period, the TIF4 schools paid out about ten retention bonuses for each recruitment 
bonus. This suggests that effective math and science teachers find retention bonuses to be meaningfully 
more compelling than recruitment bonuses that are twice as expensive and require a longer time 
commitment. Critically, these are teachers who have already shown success in meeting the needs of 
students at HISD’s hard-to-staff schools. Taken together, these two findings strongly suggest that the high 
turnover among HISD’s math and science teachers can be mitigated through investment at specific 
campuses. 


Whatever the exact reason for the increased retention among math and science teachers at TIF4 schools, 
the students of the TIF4 project schools were the ultimate beneficiaries of these teachers’ decisions to stay 
— generating statistically significant and meaningful gains in their math and science achievement during 
the grant period (Price, Christian, & Stevens, 2018). HISD staff and school leadership look forward to 
building on these takeaways as we continue to serve the diverse needs of our students and families. 
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Endnotes 


(i) Note that the figures in this report do not include ASPIRE Awards earned by campus-based support staff whose 
Awards were not supported by TIF4 funding. See details in Appendix B, Tables 1 and 2. 


(ii) For the purposes of this STEM incentive, “Core Foundation Courses” included ONLY those courses identified by 
the Texas Education Agency under the Core Foundation areas of Mathematics and Science at the elementary and 
middle school level, and those math and science Core Foundation courses required for graduation credit in the 4x4 
Recommended or Distinguished High School Diploma programs. 


(iii) These individuals were identified by their contract type and their years of previous experience: a probationary 
contract, with zero years of experience. 


(iv) In the model, the estimates are statistically “shrunk” towards the mean across the five-year window — this reduces 
the distortion that is possible from a single year that is very strong or very weak. 


(v) While HISD’s educator evaluation system makes a distinction between the two levels, both levels are considered 
“Ineffective” under the federal TIF grant reporting guidance. 
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Appendix A: Teacher Incentive Fund in HISD 


Since established by an Appropriations Act in 2006, the Teacher Incentive Fund (TIF) competitive grant 
program in the U.S. Department of Education (the Department) has supported human capital strategies for 
teachers and school leaders, “to ensure that students attending high-poverty schools have better access 
to effective teachers and principals, especially in hard-to-staff subject areas” such as science, math, and 
STEM (Science, Technology, Engineering, and Mathematics). While the specific programming supported 
through the TIF grant program has evolved since 2006 (Miller et al., 2015), TIF projects are supported by 
the Department to develop and implement sustainable performance-based compensation systems 
(PBCSs) for teachers, principals, and other personnel in high-need schools in order to increase educator 
effectiveness and student achievement. Houston Independent School District (HISD) was awarded over 
$43 million as part of the first and third cohorts of TIF grantees — $11.8 million in 2006, and $31.3 million in 
2010. A recap of these program activities is available on HISD’s website (Price & Stevens, 2017). 


In September 2012, HISD was awarded a TIF grant for $15.9 million over five years (OESE, 2012b) — one 
of just six STEM projects funded among the fourth cohort of awards (TIF4-STEM): HISD, plus Calcasieu 
Parish (LA), National Institute for Excellence in Teaching (IA), Orange County (FL), Washoe County (NV), 
and the South Carolina Department of Education. 


These grantees committed to the two Absolute Priorities required of all TIF grantees, as well as a third 

Priority that was specific to STEM programming: 

e Priority 1 (all grantees): “An LEA-wide human capital management system (HCMS) with educator 
evaluation systems at the center that (a) is aligned with the local education agency's (LEA's) vision of 
instructional improvement and (b) uses information generated by the evaluation system to inform key 
human capital decisions, such as recruitment, hiring, placement, dismissal, compensation, professional 
development, tenure, and promotion.” 

e Priority 2 (all grantees): “An LEA-wide educator evaluation system based, in significant part, on 
student growth. The frequency of evaluation must be at least annually and the evaluation rubric should 
include at least three performance levels and (a) two or more observations during each evaluation 
period, (b) student growth for the evaluation of teachers at the classroom level, and (c) additional factors 
determined by the LEA. In addition, the evaluation system must generate an overall evaluation rating 
based, in significant part, on student growth and the evaluation system must be implemented within the 
timeframe specified in Priority 2.” 

e Priority 3 (STEM grantees): “Improving STEM achievement by developing a corps of skilled STEM 
master teachers by providing additional compensation to teachers who (a) receive an overall evaluation 
effectiveness rating of effective or higher under the evaluation system, (b) are selected based on criteria 
that are predictive of the ability to lead other teachers, (c) demonstrate effectiveness in one or more 
STEM subjects, and (d) accept STEM-focused career ladder positions. In addressing this priority, each 
LEA needs to identify and develop the unique competencies that, based on evaluation information or 
other evidence, characterize effective STEM teachers. Projects also need to identify hard-to-staff STEM 
subjects and use the HCMS to attract effective teachers, leverage community support and expertise to 
inform the implementation of its plan, ensure that financial and non-financial incentives are adequate 
to attract and retain persons with strong STEM skills in high-need schools, and ensure that students 
have access to and participate in rigorous and engaging STEM coursework.” 


See _http://www2.ed.gov/programs/teacherincentive/2012-374ab.pdf for the full text of the application 
package for TIF4 (OESE, 2012a). 
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Appendix Figure 1. Geographic Location of the TIF4 Project Schools 
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Appendix B: Detailed Breakdown, ASPIRE Award at TIF4 Project Schools 


Appendix Table 1. Count of ASPIRE Awards Paid at Each TIF4 Project School, 2012-2017 


Number of Awards 2012-2013 2013-2014 2014-2015 2015-2016 2016-2017 Total 
n=18 n=18 n=20 n=16 n=14 n=23 
Blackshear ES 1D se 1) a 18 
Braeburn ES 6 29 18 3 46 102 
Burrus ES 8 2 16 4 20 50 
Codwell ES 24 24 
Durkee ES 2, 4 34 3 43 
Eliot ES 23 23 3 4 30 83 
Fleming MS w 22 lal 1 9 50 
Fondren MS 2 9 11 33 40 95 
Foster ES 9 3 3 3 14 32 
Garden Oaks Montessori 25 28 34 9 9 105 
Grissom ES 1, il 13 
Herrera ES 33 2 3 2 38 78 
Law ES il 1 2 1 5 
Looscan ES 17 7 24 
Mading ES il 1 
McGowen ES 2 18 2 22 
Milne ES 18 ‘ll 1 20 
Montgomery ES 3 3 6 
Pugh ES 16 14 1 31 
Ross ES 10 1 18 29 
Southmayd ES 82 26 il lS 26 100 
Sugar Grove Academy 2 3 4 3 2 14 
Wilson Montessori 2. 26 23 5 il 67 
Total Awards 224 207 234 91 256 1,012 


Notes: The 12 Awards paid to Dodson staff in 2012—2013 are included in the cell with the five Awards to 
Blackshear ES. These counts do not include individuals who were not eligible for TIF4 funding. The ASPIRE 
groups eligible for TIF4 funding include Group 1, Group 2, Group 3, Group 4, Group 1L, and Group 2L. The 
ASPIRE groups not eligible for TIF4 funding included Group 5, Group 6, and Group 7. For more details about 
the ASPIRE Award groups, please see additional HISD reporting on the ASPIRE Award (e.g., Hui, Mosier, & 
Bigner, 2018; Zimmerman, Hui, & Mosier, 2017). 
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Appendix Table 2. Sum of ASPIRE Award Payout at Each TIF4 Project School, 2012-2017 
ASPIRE Award Payout 


70h 40) 


n=18 


2013-2014 


n=18 


YAO} V7 0 he 40) ho tey40 ho 40) Ao ty0H Wg 


=A) 


n=16 


n=14 


Total 
n=23 


Blackshear ES 
Braeburn ES 
Burrus ES 

Codwell ES 

Durkee ES 

Eliot ES 

Fleming MS 
Fondren MS 

Foster ES 

Garden Oaks Montessori 
Grissom ES 
Herrera ES 

Law ES 

Looscan ES 
Mading ES 
McGowen ES 

Milne ES 
Montgomery ES 
Pugh ES 

Ross ES 
Southmayd ES 
Sugar Grove Academy 
Wilson Montessori 
Total Award Payout 


$61,000.00 
$8,000.00 
$45,500.00 


$15,000.00 
$52,000.00 
$55,000.00 
$15,000.00 

$4,500.00 
$19,500.00 


$109,100.00 
$10,000.00 
$18,500.00 


$14,000.00 


$5,000.00 
$119,083.33 
$15,000.00 
$21,000.00 
$587,183 


$109,066.67 
$10,000.00 


$25,000.00 
$122,333.33 
$128,500.00 
$56,575.00 
$13,500.00 
$110,750.00 


$15,000.00 
$10,000.00 
$12,250.00 


$10,700.00 
$10,000.00 
$20,000.00 
$34,000.00 


$113,500.00 
$15,000.00 
GSILIL7/ SHSIS) SIS) 
$933,508 


$7,500.00 
$56,212.50 
$24,750.00 
$16,500.00 
$155,775.00 
$18,750.00 
$57,956.25 
$60,525.00 
$15,000.00 
$98,850.00 
$27,000.00 
$22,500.00 
$7,500.00 


$3,750.00 
$19,500.00 
$3,750.00 


$27,000.00 


$7,500.00 
$26,250.00 
$74,625.00 
$731,1934 


$10,833.33 
$13,125.00 


$15,000.00 
$15,833.33 
$5,000.00 
$91,625.00 
$4,583.33 
$27,125.00 


$7,500.00 
$1,666.67 


$12,500.00 
$5,000.00 
$5,000.00 
$29,583.33 
$10,000.00 
$11,250.00 
$265,625 


$111,125.00 
$61,875.00 


$78,000.00 
$35,000.00 
$179,625.00 
$49,875.00 
$25,833.33 
$5,000.00 
$125,375.00 


$5,750.00 


$63,375.00 
$61,187.50 
$10,000.00 

$1,250.00 
$813,2701 


$68,500.00 
$295,237.50 
$155,250.00 
$16,500.00 
$210,775.00 
$286,916.67 
$281,456.25 
$403,350.00 
$87,458.33 
$282,058.33 
$32,000.00 
$279,475.00 
$29,166.67 
$30,750.00 
$3,750.00 
$35,950.00 
$27,750.00 
$32,500.00 
$66,000.00 
$73,375.00 
$330,854.17 
$76,250.00 
$225,458.33 
$3,330,781 


Note: Total Award Payments (bottom row) are rounded to the nearest dollar. These figures do not include the 
fringe benefits on this compensation. These figures combine federal (TIF4), local (cost-sharing match, and local 
supplement), and state funds ($100,517 from the District Awards for Teacher Excellence program, or DATE, for 
ASPIRE 2012-2013). The monetary value of the 12 Awards paid to Dodson staff for 2012-2013 are included in 
the cell with the five Awards to Blackshear ES. Also, these figures do not include ASPIRE Awards paid from local 
funds to individuals who were not eligible for TIF4 funding. The ASPIRE groups eligible for TIF4 funding include 
Group 1, Group 2, Group 3, Group 4, Group 1L, and Group 2L. The ASPIRE groups not eligible for TIF4 funding 
included Group 5, Group 6, and Group 7. For more details about the ASPIRE Award groups, please see additional 
HISD reporting on the ASPIRE Award (e.g., Hui, Mosier, & Bigner, 2018; Zimmerman, Hui, & Mosier, 2017). 
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Appendix C: Sample Notice of Initial Eligibility for STEM Bonus 
The following message was sent to the 59 teachers at TIF4 project schools who were notified in March 
2017 that they met eligibility criteria for the STEM retention bonus supported by the TIF4 grant. Fields in 
fixed width font and enclosed with double-angle quotation marks indicate the MS Excel fields that 
were merged in MS Word to generate and send individual communications. 
Email Date: March 24, 2017 
Email Title: Notice of Initial Eligibility. STEM Bonus for «TEACHERNAME» 
Email Body: 
Dear «TEACHERNAME» («TITLE» at «SCHOOL1516»): 
Thanks to a federal grant, select teachers in HISD can receive a STEM Retention bonus in 
September 2017 by meeting specific eligibility criteria. You are receiving this email because records 
show that you meet initial eligibility criteria for this bonus. (See Table below.) 
Should you return to a TIF4 project school to teach there for the 2017-2018 academic year, you 
will meet all criteria to receive a bonus of up to $5,000. Please take this into consideration as you 


make your plans! 


Teacher ID # «EMPL-ID»: Records 


Employed at TIF4-STEM School for 2016-2017 MI «SCHOOL1516» 
2016 Summative Rating in TADS is 3 or 4 M «TADS1516» 
2016 Subject is STEM-related M «SUBJECT» 


2016 Comparative Growth metric in the top quintile for math or science M «CMPGRO1516» 


Download and view the full award eligibility criteria here. Details about quintiles can be found at the 
[link], and information on Comparative Growth can be found at [link]. 


These retention incentives will be paid out on or before September 30, 2017. 


Please note that Principal «P-LASTNAME» has also received this information about your initial 
eligibility. Let me know if you have any questions! 


Thank you, 


Lauren E. Price 
Grant Manager, TIF4 


** NOTE: Please note that the exact amount of any post-tax award will depend on your own 
individual financial situation — as bonuses can be taxed differently than other compensation. Also, 
this notice of bonus eligibility does not confirm or contradict any current or future offer of 
employment with HISD. 
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Appendix D: Retention and Mobility of New Math and Science Teachers 


Appendix Table 3. Mobility and Retention in Three Cohorts of New Math Teachers 
Ted aleye) | Count Came Back Next Year Moved Schools Next Left District’s 
Year Cohort at Start 1) ae 2 <0) (= Yearin___ Role: Teacher Roster 


2012-13 1 604 380 71 24 12 101 
2013-14 at 487 (S\5) 63 20 16 115 
2014-15 1 356 163 70 20 16 79 
2015-16 Al 284 143 76 16 8 36 
2016-17 1 259 138 65 9 4 29 
2017-18 al 225 = = 5 2 = 
2013-14 2 654 357 70 23 13 156 
2014-15 2 464 232 82 25 Alls) oi 
2015-16 2 371 184 84 14 7 72 
2016-17 2 2909 150 69 ils} 8 48 
2017-18 2 257 - - 7 7 - 
2014-15 S 700 SEZ 74 32 20 147 
2015-16 3 518 279 93 22 12 97 
2016-17 3 420 218 80 18 10 al 
2017-18 3 338 - : : - - 


Appendix Table 4. Mobility and Retention in Three Cohorts of New Science Teachers 
Ted aLoye) | Count Came Back Next Year Moved Schools Next Left District’s 
Year Cohort at Start 1} a 3 <0) (= Yearin___ Role: Teacher Roster 


2012-13 1 547 336 61 27 11. 98 
2013-14 al 435 223) 3 23 9 94 
2014-15 1 330 152 72 16 15 67 
2015-16 al 267 Alsi 76 8 ils) 36 
2016-17 1 236 107 70 10 6 30 
2017-18 al 203 2 2 2 2 > 
2013-14 2 617 341 81 20 13 133 
2014-15 2 456 223 84 23 ile 95 
2015-16 2 357 179 79 13 7 72 
2016-17 2 286 130 79 8 9 48 
2017-18 2 241 - - 7 7 - 
2014-15 3 667 376 70 32 19 143 
2015-16 3 497 264 92 22 2 95 
2016-17 3 404 202 81 16 12 78 
2017-18 3 319 - - - - - 
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Appendix E: Same-School Teacher Retention and Student Demographics 


The school’s random effect is the school’s distance from the expected level of retention for schools with 

similar student populations. These model results were used to derive an estimate of how schools compared 

to each other in terms of in retaining each group: New and High TADS, New and Low TADS, Experienced 

and High TADS, as well as Experienced and Low TADS: 

e Teacher: Coded in the human resources dataset as teachers according to their job function and salary 
plan (PeopleSoft, and SAP OneSource), 

e New: Three or fewer years of teaching experience for a given year. 

e Experienced: Four or more years of teaching experience for a given year. 

e High TADS: Summative appraisal score of 3 or 4 in the TADS final dataset for the specific year. 

e Low TADS: Summative appraisal score of 1 or 2 in the TADS final dataset for the specific year. 


In each of these figures, the dark line represents the district-wide relationship between the likelihood that a 
teacher will return to the same campus for the following year, as a function of the school’s demographics. 
Schools whose dots are above the district-wide trend line are retaining teachers at a higher rate than 
expected for that demographic factor; schools below the trend line are retaining teachers at a lower rate 
than expected. Appendix Figures 2-4 (p. 31-32) address the retention of all teachers, while Appendix 
Figures 5-12 (pp. 33-36) address math and science. 


Schools participating in the TIF4 grant are represented by red triangles. All other schools are represented 
by blue dots. 


Appendix Figure 2. Same-School Retention for Four Teacher Categories, by School’s Economic Disadvantage 
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Appendix Figure 3. Same-School Retention for Four Teacher Categories, by School’s Percent Hispanic 
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Appendix Figure 4. Same-School Retention for Four Teacher Categories, by School’s Percent African-American 
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Appendix Figure 5. Math Teacher Retention for Four Categories, by School Effect and Percentile Rank 
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Appendix Figure 6. Science Teacher Retention for Four Categories, by School Effect and Percentile Rank 
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Appendix Figure 7. Math Teacher Retention for Four Categories, by School’s Percent Economic Disadvantage 
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Appendix Figure 8. Science Teacher Retention for Four Categories, by School’s Percent Economic Disadvantage 
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Appendix Figure 9. Math Teacher Retention for Four Categories, by School’s Percent Hispanic 
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Appendix Figure 10. Science Teacher Retention for Four Categories, by School’s Percent Hispanic 
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Appendix Figure 11. Math Teacher Retention for Four Categories, by School’s Percent African-American 
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Appendix Figure 12. Science Teacher Retention for Four Categories, by School’s Percent African-American 
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